Scope bounding with automated specification inference for scalable software model checking

ABSTRACT

A scalable, computer implemented method for finding subtle flaws in software programs. The method advantageously employs 1) scope bounding which limits the size of a generated model by excluding deeply-nested function calls, where the scope bounding vector is chosen non-monotonically, and 2) automatic specification inference which generates constraints for functions through the effect of a light-weight and scalable global analysis. Advantageously, scalable software model checking is achieved while at the same time finding more bugs.

CROSS REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional PatentApplication Ser. No. 61/420,983 filed Dec. 8, 2011 which is incorporatedby reference in its entirety as if set forth at length herein.

TECHNICAL FIELD

This disclosure relates generally to the field of computer software andin particular to a method for scalable software model checking.

BACKGROUND

Model checking and static analysis are well-known techniques for findingsubtle flaws in software systems. However, the lack of scalability andinaccuracies in the models employed leads to false positives and aretherefore detriments to their more extensive use.

Accordingly, methods which permit the more effective application ofmodel checking and model checkers to large software systems wouldrepresent a welcome addition to the art.

SUMMARY

An advance in the art is made according to an aspect of the presentdisclosure directed to a computer implemented verification method thatadvantageously exploits a particular software verification toolemployed. Advantageously, the method of the instant disclosure permitsthe more effective application of software model checkers to largesoftware systems.

Viewed from one exemplary aspect, the present disclosure is directed toa computer implemented verification method and underlying framework thatprovides a continuous or perpetual program analysis environment thatanalyzes a computer software program not once—as is common in the priorart—but continuously even after that computer software program ischanged. In this advantageous manner, the continuous framework mayanalyze the computer software program on a fixed schedule, for example,daily, weekly, etc.

Viewed from another exemplary aspect, the present disclosure is directedto a method and framework which uses information gained from prioranalysis to adaptively improve the current analysis—for exampleimproving any bounds used in scope bounded analysis. Similarly, portionsof the computer software program under analysis which have not beenaffected by changes do not require that their analysis be repeated.

Accordingly, and in sharp contrast to the prior art, methods accordingto an aspect of the present disclosure advantageously work for allproperties within a statically-determined scope. More specifically,instead of expanding the scope iteratively, a method according to thepresent disclosure refines pre-conditions and stubs for cut-offfunctions, based on witness traces, non-monotonically.

Advantageously, such methods and framework improve the scalability, andprecision while still permitting user input.

BRIEF DESCRIPTION OF THE DRAWING

A more complete understanding of the present disclosure may be realizedby reference to the accompanying drawings in which:

FIG. 1( a) shows an example function call rewriting according to anaspect of the present disclosure;

FIG. 1( b) is a schematic depicting a depth cutoff scheme on a callgraph according to an aspect of the present disclosure;

FIG. 2 shows a program excerpt containing a subtle inter-procedural bug;

FIG. 3 shows a schematic of a program analysis framework according to anaspect of the present disclosure;

FIG. 4 shows a schematic depicting hoisting annotations acrossstatements and branches; and

FIG. 5 is a schematic block diagram depicting a representative computersystem with which the method and structures according to the presentdisclosure may be implemented.

DETAILED DESCRIPTION

The following merely illustrates the principles of the disclosure. Itwill thus be appreciated that those skilled in the art will be able todevise various arrangements which, although not explicitly described orshown herein, embody the principles of the disclosure and are includedwithin its spirit and scope.

Furthermore, all examples and conditional language recited herein areprincipally intended expressly to be only for pedagogical purposes toaid the reader in understanding the principles of the disclosure and theconcepts contributed by the inventor(s) to furthering the art, and areto be construed as being without limitation to such specifically recitedexamples and conditions.

Moreover, all statements herein reciting principles, aspects, andembodiments of the disclosure, as well as specific examples thereof, areintended to encompass both structural and functional equivalentsthereof. Additionally, it is intended that such equivalents include bothcurrently-known equivalents as well as equivalents developed in thefuture, i.e., any elements developed that perform the same function,regardless of structure.

Thus, for example, it will be appreciated by those skilled in the artthat the diagrams herein represent conceptual views of illustrativestructures embodying the principles of the invention.

In addition, it will be appreciated by those skilled in art that anyflow charts, flow diagrams, state transition diagrams, pseudocode, andthe like represent various processes which may be substantiallyrepresented in computer readable medium and so executed by a computer orprocessor, whether or not such computer or processor is explicitlyshown.

In the claims hereof any element expressed as a means for performing aspecified function is intended to encompass any way of performing thatfunction including, for example, a) a combination of circuit elementswhich performs that function or b) software in any form, including,therefore, firmware, microcode or the like, combined with appropriatecircuitry for executing that software to perform the function. Theinvention as defined by such claims resides in the fact that thefunctionalities provided by the various recited means are combined andbrought together in the manner which the claims call for. Applicant thusregards any means which can provide those functionalities as equivalentas those shown herein. Finally, and unless otherwise explicitlyspecified herein, the drawings are not drawn to scale.

Thus, for example, it will be appreciated by those skilled in the artthat the diagrams herein represent conceptual views of illustrativestructures embodying the principles of the disclosure.

By way of some additional background, software model checking is awell-known technique for finding subtle bugs in software programs. Oneparticular utility of model checking and model checkers is their abilityto present witnesses to help explain a bug to a developer. As is knownby those skilled in the art, witnesses also allow the developer toquickly evaluate the validity of any bug reports and arrive at possiblefixes. Notwithstanding these virtues however, their lack of scalabilityand susceptibility to false positives due to modeling abstractions detertheir more widespread industrial adoption.

Accordingly, a practical software model checker must exhibit a number ofcharacteristics including (1) Scalability: In our experience, the modelchecker should handle 1MLOC and beyond for C source code; (2)Performance: The model checker must complete the verification within theallotted time; and finally, (3) Accuracy: The model checker shouldexhibit a low rate of false warnings, so that human effort is notwasted.

Varvel is a software verification tool based on an earlier researchprototype called F-Soft. Varvel uses a combination of abstractinterpretation and model checking to find common errors in C/C++programs including pointer usage errors, buffer overruns, C stringerrors, API usage violations as well as violation of user-definedassertions. At its core Varvel utilizes a bit-precise SAT-based boundedmodel checker (BMC) which translates the given program and properties tobe verified into a series of SAT formulae.

These formulae are solved by a SAT solver to generate witnesses thatcorrespond to property violations. Varvel has been engineered to handlelow-level aspects of C programs including pointer arithmetic, dynamicmemory allocation, function pointers, bitwise operations, structure andunion types with field sensitivity, and multi-dimensional arrays.

The scalability of Varvel is obtained in part through the use ofnumerical domain abstract interpretation using domains such as octagonsto statically prove properties. Once a property is initially provedusing a static analyzer, it is advantageously removed from furtherconsideration. Program slicing is used to remove large portions of aprogram that are irrelevant to the remaining properties, along withconstant folding to simplify complex program expressions.

Abstract interpretation is staged from “simple” inexpensive analysessuch as interval analysis to more expensive domains such as disjunctiveoctagons, with slicing and simplification used at the end of each stageto reduce program sizes. In practice, roughly 70-90% of theautomatically instrumented properties in a program are rapidlyeliminated from consideration using static analysis before the boundedmodel checker is deployed.

In spite of these optimizations in Varvel, it remains prohibitively hardto perform a whole program analysis for large programs. Moreparticularly, Varvel treats every function appearing in a given programas a possible entry function for verification. Although this helps toverify some functions, challenges remain for entry functions with largecall graphs where the models may become too large.

Notably, this can generate false witnesses. When an internal function istreated as an entry function for verification, the input parameters andglobal variables are assumed to hold arbitrary values. In reality,however, these parameters are constrained based on how the function iscalled in the program. As a result, false witnesses may be produced byassigning a value to an input that cannot be realized in an actualexecution.

In this disclosure, we present a framework called DC2 (Depth Cut-off andDesign Constraint generation) that forms part of the verificationprocedure used in Varvel. DC2 is a combination of scope bounding andautomated specification inference techniques. Scope bounding refers tolimiting the size of the generated model by excluding deeply-nestedfunction calls. Automatic specification inference refers to thegeneration of constraints for functions by a light-weight and scalableglobal analysis. The aim of DC2 is to enable scalable software modelchecking and at the same time finding more bugs.

Scope Bounding

As may be readily appreciated, the state space of a program is ofteninfinite due to recursion, heap-allocated data, loops, and so on.Consequently, bounded software model checkers artificially limit thesize of structures that might be infinite. For instance, Varvelintroduces the following bounds: the maximum size of heap to model,unrolling-depth for recursive calls, and time available forverification, etc. We call this set of bounds a scope bounding vector,each component of which can be changed independently. While the boundingmay cause the model checker to miss bugs that occur beyond the bounds,it nevertheless helps the software model checker find useful bugs inpractice.

DC2 introduces one additional bound into software model checking in anon-intrusive way. More particularly, in order to generate a model thatis amenable to model checking, DC2 cuts off function calls beyond auser-specified depth, where depth is defined as the shortest path fromthe entry function. Cut-off is done by replacing a function call with adummy-function call.

We note that in our exemplary framework according to an aspect of thepresent disclosure, the inference of pre/post conditions and stubs issupported by simple whole program analysis we call SpecTackle. Therefinement process, we call CEGER, employs an analysis around acounterexample generated by a model checker and identified as a falsealarm by the user.

The algorithm is described in FIG. 1( a). φ^(entry) and φ_(g) are theconstraints and stub_g is the stub inferred. Instrumentation isperformed as a preprocessing step that is independent of theverification itself. The depth cutoff scheme is illustrated in FIG. 1(b).

Note that the entry function of the analysis, the precondition inferredby SpecTackle, is assumed, whereas it is asserted for other calledfunctions. The function h is deemed outside the scope. Therefore, a stubreplacement—h_stub—is automatically created and the inferredpreconditions, post-conditions, and stubs are used in place of h.

Advantageously, the instrumentation does not affect the soundness ofstatic analysis. When the static analyzer encounters a call to afunction whose definition is not available (or has been cut off), ituses a conservative update to model the side-effects of the function.

To be useful in practice however, the abstract interpreter in Varvelperforms a conservative update only for the actual parameters ofundefined functions. For every actual parameter v, it assigns theabstract value

(which refers to a non-deterministic concrete value) to v·γ, where γ isthe set of access paths that consist of field and array indices. Becausethe maximum number of static or dynamic array elements to model and themaximum number of recursive field accesses are bounded, the length ofaccess paths and the number of access paths are also bounded.

Automated Specification Inference

Scope bounding largely alleviates the issue of scalability and oftenprovides enough local context within the bounded scope to find usefulbugs. However, just cutting off deep function calls is not sufficient,since it can adversely impact accuracy. For instance, inputs forinternal functions are still not constrained. In addition, theover-approximated side effects of cut-off functions may cause more falsebugs. Further, we may miss bugs that occur outside the scope.

To cope with these issues, DC2 utilizes a light-weight global analysiscalled SpecTackle. SpecTackle has two roles: (1) to generatepreconditions for functions, and (2) to generate stubs which capturesimportant side-effects of functions. Both these are tailoredspecifically to the checkers in Varvel.

Preconditions

SpecTackle visits functions in a reverse topological order with respectto the call graph of the program. For each function, it hoistsconditions within the body to the start of the function, by backpropagating the conditions across the statements. Hoisted conditions areused for two purposes: (1) to constrain the inputs of an internalfunction f when it is an entry function for verification, and (2) toassert the correctness of inputs to a called function g.

Note, in particular, that the assertion check assert (φ_(g)) willtrigger a violation during model checking and generate a witness in casethe hoisted condition g is not correct. In other words, we do notrequire SpecTackle to generate correct preconditions. In practice, wetry to automatically generate likely preconditions. The generatedwitnesses by Varvel can be additionally used for manual refinement ofthe preconditions, if desired.

Here, we briefly describe the kinds of preconditions that SpecTacklegenerates for a given function.

Pointer Validity:

For every pointer p that is dereferenced without checking for nullness,SpecTackle back propagates the condition (p !=NULL) to the beginning ofthe function. If the resulting condition involves a formal parameter ora global variable, it will be retained as a precondition for thefunction.

Array Bound:

For every variable i that is used as an index expression for accessing astatic array of size n, it back-propagates the condition (0<i<n) to thestart of the function. If the resulting condition depends only on theinputs to the function, it will be used as a precondition thatconstrains the range of the respective inputs.

Allocated Heap Size:

If a dereferenced expression is an input variable itself or is aliasedwith an input, SpecTackle tries to capture the constraints on the sizeof heap area pointed-to by the expression by analyzing pointerarithmetic operations. For example, consider the following function:

void f(T*t1,int k)

-   -   {T*t2; t2=t1+k; *t2=7;}

From the expression *t2 and the pointer arithmetic operation t2=t1+k,SpecTackle generates the following precondition for f:_heap_size (t1)>k.

Field Relation Inference:

SpecTackle captures specific patterns of field usage. For instance, weoften see struct definitions that have two fields, one pointing to theheap and the other indicating the length of the heap as below:

struct S {char*buf; size_t len;} st;

SpecTackle automatically extracts such relations and globally constrainsthe type of inputs.

Assertion Hoisting:

SpecTackle hoists user-defined assertions (written with assert) to thebeginning of functions. Moreover, Varvel allows users to writepreconditions. Those manually written preconditions are also hoisted bySpecTackle. SpecTackle is designed to be fast and scalable even forlarge software. Therefore, we avoid computing weakest preconditionssince it is typically too costly for the whole program. Although a bugmay be missed due to an incorrect assumption that was never caught atthe higher level (when checked as an assertion), we have found this notto be a problem in practice.

Hoisting of Annotations

We refer to assertions (user-defined, or automatically instrumented byvarious checkers) and preconditions as annotations—these are hoisted bySpecTackle. SpecTackle is advantageously designed to be fast andscalable even for large software systems. Therefore, we avoid computingweakest preconditions since it is typically expensive for the wholeprogram. This section briefly discusses how annotations are hoisted tothe beginning of functions and across calls in the call graph, by usingpurely syntactic domain operations.

Let φ_(l) be a precondition annotation generated at a programpoint/inside a function ƒ. Our goal is to compute an appropriateprecondition ψ corresponding to the entry point of function ƒ. To hoistan annotation φ_(l) at location l, SpecTackle initalizes l with φ_(l)and every other location with the condition false. ψ is computed bymeans of a backwards data-flow analysis that captures the effect ofvarious assignments and conditional branches between program point l andthe entry point of f.

Preconditions across assignment statements are treated by substitutionof the right-hand side in place of the LHS expression. The pre operatoris also defined to propagate a constraint φ backwards across a branchcondition c. Computing this operation involves the syntactic search fora conjunct in φ that contradicts the branch condition c. If such aconjunct is obtained, the precondition is set to false. If, on the otherhand, φ contains a conjunct that is identical to c (syntactically), theresult of the precondition is given by true. Failing this, theprecondition is set to φ, instead of the more general constraint c

φ. This is done in part to keep the syntactic form of the preconditionssimple so that the dataflow analysis can be performed efficiently.Formally,

${{pre}\left( {\varphi,c} \right)} = \left\{ \begin{matrix}{false} & {{if}\mspace{14mu}\varphi\mspace{14mu}{\,^{``}{syntactically}}\mspace{14mu}{contradicts}^{''}\mspace{14mu} c} \\{true} & {{if}\mspace{14mu}\varphi\mspace{14mu}{\,^{``}{is}}\mspace{14mu}{identical}\mspace{14mu}{to}^{''}\mspace{14mu} c} \\\varphi & {otherwise}\end{matrix} \right.$

A join operator (□) is used to merge two preconditions φ and ψ obtainedfrom the two parts of a branch. The join operator works by matching itsoperands syntactically. If one of the operand syntactically matchesfalse, the result is taken to be the other operand. If ψ can be obtainedby conjoining some assertion ψ′ to φ then the join chooses the weakerassertion φ. Finally, if the operands do not fall into any of thecategories above, the result of the join is the trivial annotation true.Formally,

${\phi\bullet\psi} = \left\{ \begin{matrix}\phi & {{if}\mspace{14mu}\psi\mspace{14mu}{is}\mspace{14mu}{syntactically}\mspace{14mu}{false}} \\\phi & {{if}\mspace{14mu}\left( {\psi \equiv {\phi\bigwedge\psi^{\prime}}} \right)} \\\psi & {{if}\mspace{14mu}\phi\mspace{14mu}{is}\mspace{14mu}{syntactically}\mspace{14mu}{false}} \\\psi & {{if}\mspace{14mu}\left( {\phi \equiv {\psi\bigwedge\phi^{\prime}}} \right)} \\{true} & {otherwise}\end{matrix} \right.$

When the analysis converges, the assertion labeling the entry point ofthe function denotes the entry precondition ψ. If ψ is an assertionother than true or false, it can be propagated to the callers of thefunction. If ψ=false, then a warning is issued to the user.

Example 1

Consider function f1 shown in FIG. 4-(a). The pointer dereference q[4]in line 3 of f1 gives rise to two annotations φ₃ and ψ₃. The bound[ptrLo(p), ptrHi(p)] represents the range of legal values for pointer p,such that p may be dereferenced in our model without causing anout-of-bounds violation. That is, ptrLo(p) represents the base addressof the memory region that contains p. In other words, accessing p withan index such that the resulting address is less than ptrLo(p) wouldcause a buffer underflow. Similarly, ptrHi(p) represents the largestaddress that is validly allowed to be dereferenced in the memory regionof p.

In the example of FIG. 4-(a), consider the assertion φ₃: q≠ NULL.Because φ₃ is identical to branch condition c: q !=NULL at line 2, theprecondition pre(φ₃,c) is true. Note that the assertion φ₄ labellingline 4 is initially fase. Therefore, joining the contribution across thetwo branches at line 2, yields φ₂: true. As a result, the annotationq≠NULL does not yield a precondition for f1.

On the other hand, hoisting the annotation ψ₃: q+4ε[ptrLo(q), ptrHi(q)]produces the precondition p+4ε[ptrLo(p), ptrHi(p)]. In a case like this,we generate the following precondition: if (p){p+4ε[ptrLo(p),ptrHi(p)]}.

Example 2

Consider function f2 shown in FIG. 4-(b). The pointer dereference *q atline 5 gives rise to the annotation φ₃. Similarly, the pointerdereference *r at line 2 gives rise to the annotation ψ₂. Annotation φ₅can be hoisted across the assignments in lines 3 and 4 yielding φ₃:p≠NULL and φ₄: r≠NULL, respectively. The join operation at the branch inline 2 yields the assertion true. As a result, annotation φ₅ does notcontribute to the precondition for f2. On the other hand, when theannotation ψ₂ at line 2 is hoisted to the start of f2, we generate thepreconditon r≠NULL for f2.

In practice, a sound and complete precondition is not strictlynecessary. If the precondition is overly restrictive, it may lead to aviolation at some call site. Similarly, if the precondition is overlyrelaxed, it may cause false alarms due to an under-specifiedenvironment. In practice, our implementation of SpecTackle sacrificessoundness in its handling of pointer indirections. Nevertheless, thenumber of unsound preconditions generated is very few in practice andsuch instances are advantageously detected through witnesses generatedby the model checker.

Stub Generation

SpecTackle generate stubs to model the side-effects of cut-off functionsusing the following analyses:

Mod-Ref Analysis:

A conservative update to all variables accessible in a cut-off functionmay generate too many false bugs. Therefore, SpecTackle conducts alightweight mod-ref (modification and reference) analysis to specifywhich variables may be modified by a function and generates a stub thatupdates only those variables that are modified within the function.

Key Effects Extraction:

Some function calls are important for verification. For instance, if afunction calls free or exit internally, it will generate a stub thatinvokes these functions.

Perpetual Program Analysis Framework

Scope-bounded program analysis systematically analyzes a program using avariety of backend solving techniques, after its state-space has beenbounded. The state-space is typically bounded using bounds in a numberof different dimensions. Some prior work has proposed, for example,bounding input-size and length of execution paths. Other typical boundsrelate to bounding the allowed heap space to be used by the codefragment under analysis, bounding the number of recursive iterations ofa function call sequence, bounding the number of loops iterations,bounding the set of function calls to consider, etc. These fixed boundsmay be iteratively relaxed in prior art to consider increasingly largerscenarios.

Most program analysis techniques however generally consider a one-timeprogram analysis environment and also consider only the expansion ofbounds in a monotonic way. A piece of software is analyzed by some toolT once, for example before code release. A human user interprets theresults of the automated analysis performed by T and marks certainwarnings as relevant warnings that need to be addressed, while othersare discarded as false warnings. Such false warnings generally occur dueto some limitation of the particular analysis used, or due to use ofunconstrained inputs that model the effect of code outside the scopeboundary. Accordingly, the present disclosure is concerned with aperpetual program analysis environment where tool T is used on afixed-schedule basis for each piece of software, such as daily orweekly.

In such an environment we are interested in:

-   -   utilizing information gained from prior analysis runs to        adaptively improve the bounds used in scope-bounded analysis,        and    -   not repeating analyses for program segments that have not been        affected by changes since the last run of T.

Adaptive scope-bounding can be used to create a sound analysis, tocreate a complete analysis, or, most commonly, to mix over- andunder-approximate abstractions to discover bugs with low false positiverates. In general, larger scopes improve the precision of the analysisbut significantly decrease the scalability.

In perpetual (continuous) program analysis, especially with a human userinvestigating warnings and classifying them as relevant or false, wehave additional information to adapt the scope bounds dynamically overtime to improve performance and precision, thereby affecting the numberand accuracy of bugs reported by the analysis. For certain bounddimensions, we incorporate a taint-based self-limitation analysis thatallows, for example, uninitialized parameters to a function that isbeyond the bounded scope horizon to be assumed safely initialized afterreturning from such a function call. The taint of such parameters to avalue representing assume-safe is propagated to others in a data- andcontrol-dependent manner.

According to an aspect of the present disclosure, we use a database ofprior runs. More particularly, for every analysis instance, we recordthe chosen scope bound parameters, we record the maximum time allottedfor the analysis, as well as the analysis results.

Analysis results include statistics of the analysis, such as time takento perform various stages of analysis, and the number of checksperformed. The checks are categorized as (1) proved, (2) tainted asassume-safe, (3) warnings concretized as witnesses, and (4) unresolved.The database used also contains detailed descriptions of the warningsthat are concretized as step-by-step witness explanation, includinginformation on data values, assumptions made at the initial state, etc.As the human user investigates the warnings, the database also recordsinformation on whether a warning was deemed relevant or false, and anyannotations provided by the user (such as interface constraints on theparameters of a function, etc.).

During perpetual program analysis, when a particular analysis instanceis about to be repeated, the database is queried to receive a high-levelunderstanding of the impact of the bound selection on prior analysisruns. With respect to the goal of the analysis, the bounds can beadaptively adjusted for the next analysis.

TABLE 1 Facts of Benchmarks Name Version ELOC #Functions thttpd 2.25b 6.7K 1.45 GenericNQS 3.50.10-prel 15.1K 253 Libupnp 1.6.6 17.9K 363Product A — 54.5K 408 Product B — 143.6K  727

TABLE 2 Verification Results for Benchmarks with and without DC2 SuccessRatio #Likely Bugs w/o DC2 w/DC2 w/o DC2 w/DC2 Thttpd 68% 96% 0 (0)  5(3) GenericNQS 82% 94% 6 (2)  6 (2) Libupnp 81% 98% 8 (0) 19 (8) ProductA 89% 96% 8 (1) 11 (6) Product B 88% 91% 14 (3)  22 (9)

For DC2 framework, we can utilize the obtained data as follows:

Adaptive Non-Monotonic Choice of Scope-Bounds:

-   -   If an earlier run failed to complete the analysis in the maximum        allotted time for a vector of scope-bounds S, we can then reduce        at least one component of the scope-bound vector S to improve        scalability, such as reducing the depth of DC2 or the size of        heap to model.    -   If an earlier run was able to complete quickly and did not        report any witnesses, but reported tainted assume-safe checks,        we can enlarge some scope to potentially find more program        errors.    -   If many warnings of an earlier run were judged as false by user,        we can automatically or manually identify relevant bounds in S        to relax for the next analysis.

It is useful to note at this time that several approaches in the priorart have used scope bounding in an attempt to achieve better scalabilityin verification systems. More particularly, Taghdiri and Jackson [See,e.g., M. Taghdiri and D. Jackson, Inferring Specification to DetectErrors in Code, ASE, 14(1):87-121, 2007] proposed acounterexample-guided refinement-based method to find bugs in Javaprograms. Their method iteratively increases the scope of calledfunctions monotonically by utilizing information from counterexamples,and continues until it finds a proof or a witness that does not rely onany unconstrained value.

Additionally Babić and Hu proposed structural abstraction [See, e.g., D.Babić and A. J. Hu, Structural Abstraction of Software VerificationConditions, CAV 2007] that monotonically increases the boundary ofverification by inserting function summaries on demand for a givenproperty.

In sharp contrast, the methods employed by DC2 according to an aspect ofthe present disclosure advantageously works for all properties within astatically-determined scope. More specifically, instead of expanding thescope iteratively, it refines the pre-conditions and stubs for cut-offfunctions based on witness traces, and chooses the scope bounding vectornon-monotonically based on stored results. In other words, the methodaccording to the present disclosure proceeds non-monotonically in sharpcontrast to the above noted methods.

Avoid Repeating the Same Analysis:

Even if an earlier run worked very well, we need to apply the analysisagain if the software to analyze has been changed since an earlier run.Advantageously, we can utilize the database to avoid repeating the sameanalysis. Existing tools can provide the difference between two versionsof software and a listing of modified functions (whose definitions havechanged). Then, if a bounded scope for a function f in an earlier runcontains any function in the modified function list, then the analysiswill be applied to function f again. In addition, if the bounded scopecontains any preconditions that have changed since the earlier run, thentoo f will be analyzed again. Otherwise, analysis off may be skipped.

EXPERIMENTS

For our experiments, we applied Varvel to five benchmarks, including twoindustry programs from NEC, with and without DC2. The description of thebenchmarks is shown in Table 1. For simplicity, we employed only a partof each benchmark for the experiments. As shown in Table 1, the columnELOC3 shows the size of modules analyzed by Varvel. Product A is adeveloper tool whose original ELOC is 100 k and Product B is a businessapplication software whose original ELOC is 1400 k.

For the experiments, the DC2 depth was set to 1, i.e., each scopeconsisted of two levels of function calls in the call graph, and we didnot manually change any preconditions or stubs generated automaticallyby SpecTackle. We conducted preliminary experiments using libupnp bychanging the DC2 depth.

We found that a depth cutoff of 1 found more bugs that a depth cutoff of0. However, depth cutoff of 2 or 3 resulted in fewer successes, i.e. themodels were too large for more functions. Note that the DC2 frameworkallows us to find a sweet-spot for scope bounding that may be differentfor different examples. We use techniques to make DC2 adaptive based onprior runs in a perpetual verification environment. As for the otherconfigurations, time bound was set to 800 s for each function, the boundof recursive field access was set to 2, and the recursive unrollingdepth was set to 1. The results are shown in Table 2. Success ratioshows the ratio of functions that were successfully verified within thetime bound. Likely bugs shows the number of witnesses that we manuallyinspected, and believed to be worth reporting to the developers.

As expected, for all benchmarks, we observed that the success ratioimproved with DC2. Clearly, scope bounding enabled application of Varvelon even those functions that had large call graphs. Further, we see thatthe number of detected bugs is also increased with DC2. Interestingly,many of the bugs found without DC2 were shallow, like dereferencing of areturn value of malloc without checking its failure in the followingstatement:

-   -   x=malloc( . . . );    -   *x= . . . ;

On the other hand, Varvel was able to find deep, interesting bugs withDC2. The number of inter-procedural bugs is reported in parentheses inTable 2, where an inter-procedural bug means a bug that spans multiplefunctions.

FIG. 2 shows an interesting inter-procedural bug in libupnp (that wasunknown), which was found only with DC2. More particularly, root_path issupposed to be initialized by config_description_doc at L1. If theinitialization fails, it is designed to jump to an error handler at L2.

However, Varvel found a case where the initialization fails and thereturned error code is UPNP_E_SUCCESS. The witness generated by Varvelindicates that if IXML_SUCCESS is assigned to err_code at L4 andixmlDocument_createTextNode returns NULL then the execution proceeds tothe error handler without changing err_code.

Note that IXML_SUCCESS and UPNP_E_SUCCESS have the same value of 0.Thus, uninitialized root_path will be given to calc_alias which assertsthe non-nullness of the corresponding parameter.

Note that this is a bug even if we do not know the definitions ofixmlNode_appendChild and ixmlDocument_createTextNode and those functionswere actually out of scope of the verification due to the DC2 depth. Weinvestigated the reasons for missing some deep bugs without DC2. Wefound that, in many cases, Varvel failed to finish static analysis orthe SAT-based BMC failed to find witnesses within the allotted time. Insuch cases, the application of DC2 is very helpful, since it reduces themodel size and the number of instrumented properties.

Without DC2, the models tend to be large and contain many properties.This makes it difficult to simplify the model using static analysis, andBMC needs to verify a large number of properties in a limited time.Thus, DC2 provided an effective solution to verify given source codewithin a reasonable time.

Turning now to FIG. 3, there is shown a schematic diagram depicting anoverall framework for performing analysis according to an aspect of thepresent disclosure. As shown in this FIG. 3, a software program underdevelopment is produced in the code development block and placed into asource code repository. Analysis are performed on this code—which may befrequently updated from the code developers—and an analysis isperformed. A separate database of analysis results is maintained whichreceives investigation inputs from users—which in turn may affect thesource code in the repository as well as affecting the analysisparameters employed.

As may be appreciated, by having a human user investigating warnings andclassifying them as relevant or false, our framework and method providesadditional information useful to adapt the scope bounds dynamically overtime thereby improving performance and precision further affecting thenumber and accuracy of bugs reported by the analysis.

One of the scope bounding techniques in our approach according to anaspect of the present disclosure involves a limit on the number offunctions whose bodies are modeled precisely. Function calls to otherfunctions are replaced by a function summary that is tainted assumesafe. To improve the number of relevant warnings found using thisapproach, we supplement the scope-bounding procedure with a global andscalable automatic specification inference tool, which we have calledSpecTackle. SpecTackle produces certain patterns of common and simplylikely pre and post-conditions for functions in a scalable fashion. Thescalability of SpecTackle is largely due to the fact thatinter-procedural pointer analysis is avoided, which may produce falsepre- and post-conditions. However, we only require the specificationsgenerated by SpecTackle to be likely true, since pre- andpost-conditions at call sites of functions are checked by theverification

We have presented a framework called DC2 that combines scope boundingand automated specification inference to achieve better scalability and,at the same time, enhance the bug-finding ability of a software modelchecker. Our experimental results for DC2 in Varvel are encouraging, andsupport our belief that a bit-precise software model checker canaccommodate the requirements from industry by carefully designing andengineering its application.

Those skilled in the art will recognize that our description providedherein is merely exemplary and additional variations to our teachingsare possible and contemplated. For example, the methods and structuresdescribed and used according to the present disclosure may beconveniently implemented on a contemporary computer system such as thatshown schematically in FIG. 5. As a further example, an adaptive DC2framework to tune the scope-bounding for given software, based onprogram metrics and prior verification runs are within the scope of ourteachings which should only be limited by the claims appended hereto.

The invention claimed is:
 1. A computer implemented method for adaptive,scope bounded verification of computer software programs comprising: a)performing an automated pattern-based specification inference on a wholecomputer software program to compute likely function pre-conditions andfunction post-conditions; b) generating multiple analysis instances fromthe whole computer software program by selecting functions to be treatedas entry points for an analysis; c) choosing a scope bound vector for aninstance using previous results stored in an analysis results recordsdatabase; d) performing the analysis for a bounded instance; e) storinganalysis results in the analysis results records database; and f)periodically performing the above steps a-e; wherein the scope boundvector is adaptively chosen non-monotonically after each iteration andwherein the analysis is performed such that the computed likely functionpre-conditions and function post-conditions are weaved into the analysisinstance at the function scope boundary.
 2. The computer implementedmethod of claim 1 further comprising: generating operator-initiatedwarnings from the results stored in the analysis results recordsdatabase and initiating investigations.
 3. The computer implementedmethod of claim 2 further comprising: updating a source code repositorywith updated computer software program code such that a new analysisinstance is generated.
 4. The computer implemented method of claim 2,further comprising: modifying the function pre-conditions andpost-conditions for the next iteration based on the analysis results andgenerated warnings.
 5. The computer implemented method of claim 1,wherein the pattern-based specification inference is based onsyntactically matching expressions in the code and handling pointerindirections.
 6. The computer implemented method of claim 1, wherein thespecification inference hoists annotations based on assertions inside afunction to its start and across function calls.
 7. The computerimplemented method of claim 1, wherein the specification inferencehoists annotations based on potential runtime errors inside a functionto its start and across function calls.
 8. The computer implementedmethod of claim 7, wherein the specification inference hoistsannotations for potential runtime errors, including segmentation faults,buffer overflows, and memory leaks.